Historically, the determination of the quality of video material has been via the subjective impression of a test audience of "non-expert" viewers. This testing has involved various CCIR recommendations for screen size, viewing distance, viewing height, ambient lighting and calibration of color temperatures of the test monitors. The results have been expressed in the CCIR-500 scale of five levels:
5=Imperceptible (in terms of noise) PA1 4=Perceptible, but not annoying PA1 3=Slightly annoying PA1 2=Annoying PA1 1=Very annoying
However, different persons will interpret noise, color, hue, brightness and contrast parameters quite differently. Hence the subjective rating of the material will vary greatly over a group of viewers even in the same viewing conditions.
Recent efforts to transmit video over limited bandwidth mediums, such as telephone lines, switching and multiplexing equipment have shown that it is likely to require sacrifices and/or tradeoffs in the quality of the video material. For example, it has become customary for video material to undergo compression in order to obtain compressed video material which may more easily be transmitted over limited bandwidth mediums. Therefore, the quality of the video delivered by a transmission system to an end user is limited by the quality of the compressed video material that it is input into the transmission system.
The Motion Picture Experts Group (MPEG) have developed frame oriented compression standards called MPEG1 and MPEG2 in which redundant information is encoded once and updated as the information changes. Subjective testing of the MPEG1 algorithm using the five category method of the CCIR 500 scale has indicated that the MPEG1 encoded video is generally less acceptable than broadcast NTSC video, and only slightly less acceptable than video produced by a consumer grade VHS type VCR. MPEG2 encoded video was found to be generally equal to broadcast quality video. However, MPEG is merely a standard format and does not guarantee the level of quality in an MPEG encoded video signal. There are a number of different compression algorithms which satisfy MPEG transmission requirements, but which may nevertheless provide encoded video of substantially different quality.
Furthermore, the production of digitally encoded video is typically a multi-step process, illustrated in FIG. 1, of capturing source material, digitizing that material and encoding it using a compression algorithm such as MPEG1 or MPEG2. Notwithstanding the emergence of video compression standards such as MPEG, compressed and encoded video may easily have significantly different levels of quality. The encoding process is even more likely to result in digital video having poor quality when the quality of the underlying analog video has deteriorated to the point shown in FIG. 5 thereby necessitating that the digitized video must be restored in addition to being encoded.
Traditional analog domain testing, such as short haul, medium haul, long haul, EIA-250 testing with color bars and wave forms testing, does not indicate how a digital network is working because it is geared toward an analog stream instead of a digitally encoded stream. For example, a color bar might be input into an encoder and come out with a different color which can then be measured. But such a test does not measure the capability of the encoding process to deal with the problems normally associated with highly compressed digital video such as detailed information and motion. Such defects are significant since motion in compressed video is a much more difficult characteristic to capture than anything else.
Known video encoder testing equipment use the H.261 standard for video conferencing and employ algorithms for the tester which are somewhat subjective. More importantly, known video encoder testing equipment is based on analog domain testing and does not test in the digital domain.
Efforts have been made to obtain a quantitative system in which findings in the CCIR-500 scale can be replicated using digital measurements. A company called StellaCom has a prototype system in which they evidently test the viability of a decoded signal based on H.261 by showing differences between the original video and the decoded digital video. However, the system is still believed to be less than successful since there is no way to take measurements while the video is still in the digital domain. The video always has to be converted back into the analog domain to take the measurements.
Therefore, a quantitative measurement system for digitally encoded video is believed to be desirable in order to effectively and efficiently test and accept a large amount of encoded video material.
Nevertheless, the primary concern of known testing systems has been in determining the quality of virgin encoded video which is derived from non-anomalous baseband video using a single compression algorithm and which is tested without first being transmitted in a distribution system. Even if the virgin video is of high quality, it may be compromised when it is input into or utilized in a digital network since the digital network itself may adversely affect the quality of the encoded video and introduce errors into the digital video signal received by the end user.
For example, asynchronous transfer mode (ATM) has been developed as a technique to provide broad-bandwidth, low delay, packet-like switching and multiplexing of digital video in backbone networks. In ATM, usable capacity can be assigned dynamically (on demand) by allocating bandwidth capacity to supply fixed-sized information-bearing units called "cells" to point-to-point or multi-point outputs. Each cell contains header and information fields. The ATM standard, CCITT.121/2 specifies a 53 byte cell which includes a 5 byte header and a 48 byte payload. Unfortunately, much of the equipment in a network, such as settop boxes and other customer premises equipment, cannot accept ATM and utilize an MPEG (moving picture experts group) standard for digital video program compression.
It is therefore necessary to adapt MPEG encoded video data into data in the ATM cell format and to then readapt the data in the ATM cell format back into MPEG encoded video data. The process of adapting MPEG-2 data into ATM cell format, however, imposes additional overhead requirements that reduce the information-carrying capacity of the network. Furthermore, the network may encode video program material into channel streams of say, 6 Mbits/sec, and packetize a number of 6 Mbits/sec channel streams into a single higher rate signal transport stream. Synchronous transmission protocols, such as SONET, may require a stream of continuous data to retain synchronization. Thus, an ATM data stream carrying MPEG video data that is transmitted on a synchronous carrier, even if padded with ATM idle cells, or "dummy cells", may not be able to maintain proper synchronization with the physical layer.
In a large network, there is immensely complicated provisioning, such as VPI/VCI provisioning, trunk assignment provisioning, etc., which has to be done correctly in order for video connections to be error free. It is quite possible to have an encoder which operates correctly and a decoder which operates correctly, but have something somewhere in the network that is not configured correctly and thus interrupting the video signal. It is very difficult to locate the point in the network which is causing the error.
It is also quite possible that the virgin video coming out of encoder and input into a complicated digital network might be severely degraded in quality when it is received by the end user after transversing the network. However, it is not possible to easily determine the part of the network which is causing the degradation because of the different formats and protocols used throughout the network. For example, the output of an encoder is frequently not in the same format as the input of a settop box even though they are both used in the same network. Therefore, it is not possible to take the settop and simply hook it up to an encoder to see if the encoder and the settop work properly when the other parts of the network are removed.
In general, the encoder and the communication path that the entire network runs over hinges on an interface between the network and the end user's settop or DET. This interface is the interface through which all of the data traffic has to flow. Although the formats and protocols that apply to the outputs of the video information providers, encoders and multiplexer at the head end of the network and the settop box at the bottom end of the network are normally standardized, the interface may be peculiar to the network and there may be certain communication parameters which must be set in that stream for the network interface module to be able to decode all of that information.
Recent improvements have further complicated the testing of compressed digital video in networks. For example, early encoding of video programs frequently took long periods of time and consequently had to be conducted off-line and provided to the network in a pre-encoded, compressed, format. The process of encoding the video data in such off-line situations, however, made it possible to manage and optimize the compression algorithm through non-linear or recursive techniques such as pre-processing, post-processing, and store and forward processing, thereby improving the quality of the resulting encoded video. It also was not objectionable if the encoding process did not output encoded video at a constant rate since it was possible to compensate for time differences after the encoding process was finished and to furnish a stored video file.
The recent introduction of "real time" MPEG 2 encoders now permits the immediate distribution of video which is not previously encoded and live action video such as sporting events, etc., over limited bandwidth networks. Although referred to as "real time" encoders, there is inevitably some sort of delay introduced when video is captured and processed in order to compress it and to digitize it. The delay may be as high as two seconds and may be even longer if more functionality is added to the encoder. The delay is also increased by whatever other delay the network imposes, and the total delay will be delivered to the television set. The acceptability of such delays might depend on the video content. Delays might be unacceptable, for example, in the video transmission of horse races to off-track betting locations. On the other hand, a long delay would be inconsequential when transmitting a movie.
Typically, a real time encoder is not able to store the video in order to do further processing on it and all the processing is done on the video only once as it's going through the system in a single pass. This and other constraints imposed on processing required to capture video, immediately encode and output encoded video at a constant rate severely handicaps the quality of the encoded video which is produced when compared to the encoded video produced by an off-line stored file.
Furthermore, a stored MPEG 2 video stream may differ in material respects from a real time MPEG 2 video stream because MPEG 2 relies upon program clock references (PCRs) to provide proper timing for the encoded video signal and avoid the problems of flickering. It is more difficult for real time encoders to produce an encoded video signal having the proper timing. The program clock references are more likely to be irregular in a real time encoded video signal than in a store and forward video signal.
There may also be compound compression problems when video is transmitted twice before it is received by the viewer. For example, video material may be forwarded from a program source such as HBO to a local distributor by compressing the video, sending it up to a satellite, bringing it back down, and decompressing it. The local distributor then compresses the video again, perhaps using a different algorithm, and distributes the video locally, which is then decompressed a second time by the each end viewer's equipment. So there may be compound problems caused by a first compression and decompression process occurring in front of a second compression and decompression engine. A defect or anomaly in compressed video may initially be minor or insignificant, but become quite significant when the video is exposed to a compression a second time.